The Use of Monetary Incentives to Increase the Reliability of the IAT Measure of Racial Bias

The Implicit Association Test (IAT) has emerged as the most common procedure to measure implicit attitudes, and in particular stereotypes towards members of an out-group. An important reason behind the popularity of the IAT is that it displays relatively high reliability when compared to other measures of implicit attitudes. Still, despite its relative merits when compared to available alternative implicit measures, an important literature in psychometrics underscores the IAT’s susceptibility to measurement error. By using such a small unit of measurement (i.e. milliseconds), changes of a tenth of second can have a highly consequential effect on a person’s IAT score. Such sensitivity can bring about substantial measurement error. Moreover, there is also a concern that the IAT may be too demanding for subjects in terms of effort, focus, and engagement. This can lead to substantial heterogeneity in subjects’ effort level which can compound measurement problems in IAT data. This paper attempts to reduce these measurement issues in IAT scores by introducing a performance-based monetary incentive. In our experiment, all subjects take a standard race IAT (where, as is always the case in IAT studies, participants are asked to “quickly and accurately” perform the “sorting tasks” that comprise the IAT). Only some subjects, however, are randomly assigned to a treatment condition, which informs them that they are eligible to earn extra money depending on how “quickly and accurately” they perform the “sorting tasks” in the IAT. By incentivizing subjects to maintain high effort level throughout the length of the sorting tasks, our design provides a simple method for reducing measurement error and generating a more reliable IAT. We then show, via a simple empirical exercise, how these measurement gains improve the predictive validity of the IAT in that they allow researchers to more easily recover potential associations between IAT scores and other constructs of interest, such as explicit racial attitudes.